Your Mark Is My Dirt: Impact of Email Signatures on Decision Making

In order to text mine email data it is important to address the substantial amount of noise usually contained in the data. This noise can skew the results of data mining and so reduce the effectiveness and efficiency of decision support systems that use these techniques. Ideally the noise is removed in pre-processing. The paper presents a case study of a series of steps to progressively clean a set of email data, and examines the negative effects that a seemingly innocuous component of email, a signature, can have when email is repurposed for text mining. The paper advocates trying to understand the consequences of noise on the mining process through small scale pilot studies as a way to gain a better understanding of the real meaning of the results of mining algorithms. This approach also serves as an exemplar for understanding the implications of variations in input patterns for the interpretation of the results data mining.